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1. Study the invariance in the shift in control strategy from continuous control 
to move-and-wait as a function of latency and control difficulty. 

2. Develop a theoretically understood way of introducing measureable levels of 
control difficulty using misalignment between display and control axes . 

(From an explanation of some results of Experiment 1) 

3. Use this understanding to study the interaction of control difficulty and 
system latency. 

4. Conducted Experiment 3 to calibrate our imposition of control difficulty 
using more representative control rotations. 

5. Conducted Experiment 4 to directly study interaction of control difficulty 
with latency. 

6. Results suggest a way to generalize latency requirements across control 
difficulties that should aid establishment of standards for managing latency 
in teleoperation. 

*NASA Human Research Program Investigators' Workshop, February 12-14, 2013,Galveston, Texas 
** San Jose State University Foundation 
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Measuring and Explaining the Misalignment Effect Function 
(MEF) and its Interaction with System Latency 
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Explaining some geometric bases of user control difficulties caused by generalized 
misalignments between display and control axes as occur during teleoperation. 

1 . Project Review 

2. Misalignment Effect Function (MEF) 

3. Definition and measurement of a spatially generalized MEF 

4. Features of the generalized MEF 

5. Control rule to explain the first part of the MEF 

6. Interactions with time delays 

7. A geometric approach to additional trajectory analyses 
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Misalignment Effect Function Definition (MEF) 


The Misalignment Effect Function describes the relative 
decrement in interaction with spatially presented objects 
due to a rotation between a user’s display coordinates and 
those with which they control the object. 



Measurement Issues 


Goal: to determine the geometric basis of control difficulties caused by the 
unusual motion due to rotated control coordinates. 

- Predict control difficulty for arbitrary control rotations & movement directions 
-Use this understanding to stress control strategies that shift with latency 

• Environmental 

- Study generalized movement and rotation and rotation to avoid isotropic bias 

- Avoid effect masking due to environmental or other cueing 

• Participants 

-Use user-produced ’’natural” placement movements 

- Account for potential large inter-subject ability differences 

- Minimize or mitigate training and sequence effects 

• The dependent measures 

- Select task with accessible geometric structure 

- Choose measure with potential practical significance in closed loop response 

- Identify a natural, theoretically based measure if possible 
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Most Natural Response Measure 
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Principal Analytic Element 



Targeting rule: Set current kinesthetic direction, x i+1 equal to movement start visual direction, x i+1 

nPL = i Ixjl / dj and lxjl« dj so 2 Ixjlcos(tp) - dj and nPL - l/cos(\p) 


Misalignment Effect Function : All Axes 


Normalized Path Length 
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Experiment 3 with more Representative Rotation 


Subjects selection/screening: < V stereo resolution, Measured IPD 

10 S, 20-64 yrs, 8 M, 2 W, unpaid x 

Task: Movement of spherical cursor to touch spherical targets randomly positioned at 
varied distance in octants surrounding start point with constant Fitts’ Index of Difficulty 

Hardware & software: High performance Rockwell-Collins SR80™ HMD-based 
customized, calibrated Virtual Environment system, head/hand tracking® 120Hz, 25 ms 
measured stable system latency, 60 Hz stereo rendering. In-house software developed by R. 
Jacoby using C++ and WorldToolKit 

Familiarization: 15-25 minute familiarization to establish baseline “normal” movement, 
inter pupilary distance adjustments, verification of stereo, task understanding, movement 
goal & task elements. 

Design and Implementation: Independent variables: “Rotation representativeness”: Yaw: 
±45°, Pitch:+35°, Roll: 0, ±30, ±60, ±90°, Blocks of fixed rotations of 10 movements 
including 3 initial, excluded warm up movements, Near & Far targets: 11 cm, 22 cm. 
Targets in all possible octants, Same targets for all, Randomized blocks design. 

Planned Dependent Measures: Normalized Path Length 

Analysis: Rotation X Target_Distance X Subjects, ANOVA 



More Generalized (Complex?) Rotation 

Euler Rotations Embodied in a Helmholtz Gimbal 
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Experiment with More Generalized (Complicated ?) Rotation 3 



All subjects get same targets in random orders. Fitts' ID = 3.3 


Experiment 4 with Time Delay 

1. Single target distance 16.5 ±5% cm 

2. 18 targets per Block 

Three warm-up targets 

14 randomly ordered targets on approximate symmetry axes of cube (± 7°) 

1 marker for manual latency change 

2. Three stress/difficulty levels: 

No rotation, 

Level 1 ±45°, 35°, ±0 (yaw, pitch, roll) 

Level 2 ±45°, 35°, ±90 

3. Five delay increments (above system minimum ~30 ms) 

0, 0.2, 0.4, 0.6 0.8 (s) 

4. Adjusted Fitts ID=4.4 to bracket presumptive strategy shift and control fatigue. 

5. Randomized selection/ordering of latency-difficulty pairing for each 18 target block 

6. Experimental Phases 

Screening for stereo and IPD measurement 
HMD and task familiarization 

Five blocks of 14 experimental targets: 0 latency increment and 0° rotation 

Twenty blocks with randomly ordered 5 levels of latency 

Forty randomized of blocks of pairing 3 difficulty levels with 5 latency levels 

7. Three-four hours over 2-3 days 
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Experiment 4 with Time Delay 2 



re 

Q_ 

~o 

Q) 

N 


re 

E 


o 


c 

re 

re 


1 


10 

8 

6 

4 

2 

0 


12 


10 



2 


0 




0 0.2 0.4 0.6 0.8 




0 0.2 0.4 0.6 0.8 


Latency (s) 


Latency (ms) 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


Experiment 4 with Time Delay 3 
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Summary 

1 . Accurately measured and partially explained the Misalignment Effect Function 
(MEF) arising in manual telerobotics.. 

2. A targeting model in which users move to a target current making kinesthetic 
direction equal to current visual direction leads to a simple, spontaneously 
arising, (new?) geometric law of part of the MEF that requires no free parameters. 

3. MEFOP) = 1/cosOP) ± < 5 % , 0 < W < 65° defines a sec(W) rule for an 

equivalence class over movement velocity profiles. 

4. Used understanding of the MEF and an experiment with more representative control 
rotations to select control difficulties to examine the invariance of control strategies 
shifts during increase latency. 

5. Conducted an experiment revealing a multiplicative interaction between latency and 
control stress due to rotation 

6. This finding provides a possible means to generalizing latency requirements across 
increasing task difficulty. 

7. Further geometric analysis of the spherical geometry of rotation provide geometric 
basis for explaining counterintuitive performance effects in teleoperation. 

8. Have identified control strategy shifts due to the interaction of latency and control 
stress but need further trajectory analysis to completely characterize it. 

9. Found some evidence in generalized movement consistent with Crossman’s 
successive approximation approach to explaining Fitts law. 



